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Gyroscopic  Many  Body  Problems  in  Cooperative  and  Adversarial  Control 

ABSTRACT 

In  this  project,  the  primary  aim  is  to  advance  the  science  of  controlling  collectives  of  systems  engaged  in  cooperative  and  adversarial 
behaviors,  by  focusing  on:  (a)  robust  dyadic  (i.e.  pair-wise)  interaction  laws  between  particles  (modeling  UAV/UGV/UUV  interactions)  to 
achieve  a  rich  collection  of  spatio-temporal  patterns  of  cooperative  behavior;  (b)  robust  scalable  dyadic  pursuit  laws  as  building  blocks  of 
adversarial  interactions;  (c)  optimality  principles  underlying  interactions  laws  that  gives  rise  to  cooperative  and  adversarial  behavior;  (d) 
insight  from  biological  processes  such  as 

prey-capture  behavior  or  pursuit,  in  echolocating  bats,  and  in  visual  insects  such  as  dragonflies;(e)  verification  of  the  theoretical  advances  in 
a  laboratory  test-bed  consisting  of  commercial  robotic  platforms  equipped  with  distributed  agent-based  control  software,  and  an  indoor  GPS 
system  based  on  ultrasonic  ranging.  Methods  used  in  this  research  include  techniques  from  geometric  mechanics  and  control  on  Lie  groups, 
and  optimal  control  and  game  theory.  The  project  has  contributed  basic  results  in  pursuit  laws  as  building  blocks  for  cooperative  and 
adversarial  control.  In  collaboration  with  biologists,  methods  developed  in  this  project  are  enhancing  our  understanding  of  prey-capture 
behavior  in  single  and  multiple  echolocating  bats. 
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Problem  Studied 


The  main  problems  of  this  project  are  (a)  To  determine  robust  scalable  dyadic  (i.e.  pair-wise) 
interaction  laws  between  particles  (modeling  UGV/UAV/UUV  collectives)  to  achieve  complex 
patterns  of  cooperative  behavior;  (b)  To  determine  robust  scalable  dyadic  pursuit  laws  that  would 
serve  as  building  blocks  for  adversarial  interactions;  (c)  To  uncover  optimality  principles 
underlying  interaction  laws  that  give  rise  to  cooperative  and  adversarial  behavior;  (d)  To  draw 
insight  from  biological  processes  such  as  prey-capture  behavior  in  individual  echolocating  bats, 
adversarial  prey-capture  encounters  in  multiple  echolocating  bats,  and  aggressive  territorial 
maneuvers  in  visual  insects  such  as  dragonflies  that  employ  stealth  mechanisms  in  their  flight 
patterns;  (e)  To  exploit  pursuit  laws  in  cooperative  behavior,  such  as  cyclic  pursuit;  (f)  To  verify 
the  theoretical  advances  made  in  attacking  problems  (a)-(c),  and  (e)  by  exploration  of  control 
laws  in  a  laboratory  test-bed. 

Methods  based  on  geometric  mechanics  and  control,  Lie  group  theory,  and  stochastic  differential 
equations  are  currently  used  in  the  project  to  attack  the  stated  problems.  Methods  from 
differential  game  theory  and  evolutionary  game  theory  have  also  been  introduced  into  our 
program  of  research.  Evolutionary  game-theoretic  methods  have  been  used  to  set  up  a 
competition  between  strategies  and  to  uncover  certain  optimality  properties  of  a  particular 
strategy,  the  motion  camouflage  pursuit.  Other  pursuit  strategies  such  as  constant  bearing  pursuit 
and  classical  pursuit  have  been  investigated  and  compared  with  motion  camouflage  pursuit.  In 
collaboration  with  biologists,  mathematical  methods  developed  in  this  project  are  enhancing  our 
understanding  of  the  prey-capture  behavior  of  echolocating  bats.  A  mobile  robot  test-bed  is  used 
in  carrying  out  a  modest  set  of  experimental  verifications  of  the  theoretical  work.  This  test-bed 
includes  a  location  sensing  network  employing  ultrasound  and  RF  transceivers  for  ranging,  in  a 
manner  akin  to  how  echolocation  is  accomplished  in  nature. 

Scientific  Barriers 

At  the  start  of  this  project,  we  faced  a  series  of  scientific  barriers  associated  with  the  role  of 
delay  and  stochasticity.  In  communicating  cooperative  collectives,  communication  delays  as  well 
as  control  delays  contribute  to  possible  instabilities.  Additionally,  stochasticity  in  the  form  of 
sensor  noise  as  well  as  actuation  noise  in  cooperative  systems,  and  randomized  strategies  in 
adversarial  systems  are  a  source  of  complexity,  and  contribute  to  perfonnance  limitations.  In 
adversarial  situations  such  performance  limitations  may  be  exploited  (for  instance  in  evasive 
behavior).  Delay  and  stochasticity  are  important  issues  in  effective  realization  of  feedback  laws 
in  neuronal  and  biomechanical  substrates.  They  are  also  important  issues  in  technological 
implementations.  At  the  start  of  this  project,  we  also  were  in  the  early  stages  of  developing  a  3- 
dimensional  theory  for  pursuit  laws  in  adversarial  systems,  with  details  remaining  to  be  worked 
out.  Additional  barriers  include:  (a)  limited  understanding  of  the  issues  of  sensory  jamming  when 
multiple  members  of  a  collective  use  active-sensing  (e.g.  bat  vocalizations)  in  a  swann;  (b) 
limited  understanding  of  selective  attentional  mechanisms  that  would  enable  successful 
exploitation  of  dyadic  interactions  in  cooperative  and  adversarial  control;  (c)  limited 
understanding  of  the  prospects  for  scalability  of  pursuit  strategies  to  collectives  of  many  agents. 


Significance 

The  central  theme  of  this  project  to  advance  the  conceptual  and  analytical  foundations  of 
gyroscopic  many  body  problems  is  an  important  one  not  only  for  its  pure  scientific  interest  but 
also  for  the  applicability  of  results  emerging  from  this  project  to  the  steering  of  collectives  of 
engineered  systems  (such  as  unmanned  aerial  vehicles  and  unmanned  ground  vehicles)  of 
interest  to  the  Army.  The  infonned  use  of  biological  inspiration  through  close  collaboration  with 
biologists,  and  extensive  access  and  familiarity  with  the  data  arising  from  prey-capture  behavior 
of  echolocating  bats  (collected  in  the  auditory  neuroethology  laboratory  of  Cynthia  Moss)  is  a 
distinctive  aspect  of  this  project.  It  is  appropriate  and  significant  due  to  the  strong  functional 
similarities  between  the  natural  system  (bat)  and  engineered  systems  (such  as  small  unmanned 
aerial  vehicles)  in  adversarial  settings.  The  exploitation  of  commercial-grade  distributed  software 
environments  for  demonstrations  of  principle,  and  the  introduction  of  ideas  and  techniques  from 
evolutionary  game  theory  for  comparisons  between  alternative  control  strategies  are  recent 
advances  in  the  project  that  promise  greater  relevance  of  this  work  for  army  missions.  The 
approach  in  this  project  has  the  distinctive  character  of  being  strongly  founded  on  principles  of 
mechanics  (second  order  dynamics),  as  opposed  to  pure  kinematics  (first  order  dynamics). 

During  2009,  we  initiated  an  international  collaboration  with  a  group  of  distinguished  physicists 
from  Italy  engaged  in  observations  of  highly  dynamic  flocking  behavior  in  large  flocks  of 
starlings  (composed  of  many  thousands  of  individuals).  Using  our  techniques  of  pursuit,  we 
worked  on  analyses  of  flock  trajectory  data,  with  the  aim  of  developing  control  laws  compatible 
with  the  topological  interactions  observed  and  investigated  by  the  Italian  researchers. 

Accomplishments 

Motion  camouflage  is  a  stealth  strategy  observed  in  nature  in  the  aerial  territorial  battles  of 
dragonflies  which  are  highly  visual  insects  capable  of  sensing  optic  flow,  but  are  not  very 
sensitive  to  looming  cues.  Early  in  this  project  it  was  observed  that  there  is  an  equivalence 
between  this  stealth  strategy  (Justh  and  Krishnaprasad,  2006),  and  a  prey-pursuit  strategy  in 
echolocating  bats  (Ghose,  Horiuchi,  Krishnaprasad  and  Moss,  2006).  Motion  camouflage  is 
geometrically  indistinguishable  from  the  pursuit  strategy  known  as  constant  absolute  target 
direction  (CATD)  pursuit  observed  in  bats  engaged  in  prey-capture.  The  latter  is  familiar  to 
missile  guidance  specialists  under  the  name  of  parallel  navigation.  This  understanding  pennitted 
the  development  of  3D  pursuit  control  laws  (Reddy,  Justh  and  Krishnaprasad,  2006),  that  are 
biologically  plausible. 

The  problem  of  sensing-actuation  feedback  delay  is  important  in  nature  and  in  engineered 
systems.  Data  from  the  echolocating  bat  showed  that  this  delay  is  in  the  vicinity  of  a  little  over 
100  msec.  We  extended  our  theoretical  work  on  motion  camouflage  to  analyze  pursuit  in  the 
presence  of  sensorimotor  delay.  We  have  shown  that  motion  camouflage  (or  pursuit  to  capture) 
is  achievable  under  suitable  hypotheses  on  delay  and  feedback  gain  (Reddy,  Justh  and 
Krishnaprasad,  2007).  The  structure  of  delay-gain  feasible  set  for  effective  pursuit  in  2D  was 
elucidated  through  detailed  analysis  in  the  Master  of  Science  Thesis  of  Viswanadha  Reddy.  In 
recent  work,  we  have  extracted  actual  curvature  and  speed  signals  associated  with  bat  flight 
trajectories  and  compared  these  to  our  theoretical  feedback  laws  for  the  setting  of  sensorimotor 
delay.  The  results  show  strong  correlation  for  a  delay  of  about  1 12  msec.  We  have  developed  an 


optimization  algorithm  for  extraction  of  curvature  and  speed  signals  from  noisy  flight  data,  based 
on  a  regularization  approach  that  penalizes  very  rapid  changes  in  curvature  or  speed.  These 
results  are  being  readied  for  publication  (Reddy,  Ghose,  Justh,  Horiuchi,  Krishnaprasad  and 
Moss,  -  in  preparation).  Work  on  further  developments  along  these  computational  lines,  using 
the  maximum  principle  of  Pontryagin  from  optimal  control  theory  is  being  pursued. 

In  nature  and  in  engineered  systems  involving  adversarial  encounters  (e.g.  missile  guidance), 
noise  in  actuation  is  important.  An  evader  (prey)  may  use  noisy  or  unpredictable  steering 
maneuvers  to  confound  and  possibly  escape  a  pursuer  (predator).  Given  the  geometrical  structure 
of  our  models,  we  asked  and  answered  the  question  of  what  the  appropriate  noise  models  might 
be.  We  also  allowed  for  jump  processes  in  the  noise.  We  further  developed  a  result  on  weak 
accessibility  of  motion  camouflage  in  the  presence  of  noise  (Galloway,  Justh  and  Krishnaprasad, 
2007). 

In  ongoing  work,  we  have  developed  a  differential  game  formulation  of  motion  camouflage 
(with  Kevin  Galloway),  which  form  a  part  of  Galloway’s  Ph.D.  work  (Galloway,  2008).  An 
evolutionary  game-theoretic  model  for  comparison  of  pursuit  strategies  (Wei,  Justh  and 
Krishnaprasad,  2009),  has  yielded  a  new  way  of  looking  at  competition  between  different  pursuit 
strategies.  In  a  recent  collaboration  with  Chen  Chiu  and  others  (Chiu  et.  al.  2010),  new  insights 
on  competitive  prey-capture  behavior  in  echolocating  bats  have  been  obtained,  using  analysis 
methods  developed  in  this  project.  Further  progress  has  been  made  in  cyclic  pursuit  in  3 
dimensions  (Galloway,  Justh  and  Krishnaprasad,  2010),  and  optimal  control  in  coupled  systems 
(Justh  and  Krishnaprasad,  2010a  and  2010b). 

In  collaboration  with  undergraduate  researchers  in  the  Intelligent  Servosystems  Laboratory,  we 
have  extended  our  test-bed  for  investigation  of  pursuit  laws  in  mobile  robotic  systems  based  on 
Cricket  an  indoor  localization  system.  The  specific  extensions  include  integration  of  odometry 
with  INS,  GPS,  and  Cricket  by  means  of  filtering  and  estimation  algorithms.  This  work  involved 
3  undergraduates  during  summer  2006,  fall  2006  and  spring  2007.  Three  additional 
undergraduates  joined  the  team  in  June  2007  for  the  summer  and  refined  the  filtering  and 
estimation  algorithms,  and  their  implementation.  They  also  carried  out  demonstrations  of  this 
work  on  mobile  robots  executing  pursuit  strategies.  In  summer  2008,  two  undergraduates,  Scott 
Livingston  (University  of  Tennessee)  and  Laura  Freyman  (University  of  Maryland),  obtained 
extensive  data  on  the  interaction  of  a  bat  in  flight  with  obstacles  in  a  flight  room  during  the 
course  of  pursuit  of  an  insect.  They  then  analyzed  this  data  from  the  viewpoint  of  geometric 
models  of  the  current  project  to  examine  hypotheses  on  correlations  between  geometric  features 
of  obstacles  (nets)  and  bat  trajectories.  The  work  received  the  top  prize  for  best  project  in  the 
summer  research  program.  During  summer  2009  two  undergraduates,  Michael  Kuhlman  (from 
Rensselaer  Polytechnic  Institute)  and  Katherine  McRoberts  (from  Grove  City  College), 
supported  under  the  same  NSF  program  for  summer  research,  developed  experimental 
implementations  of  control  laws  for  collision  avoidance  using  a  sonar  sensor  signal  processing 
system  (due  to  T.  K.  Horiuchi)  inspired  by  bat  sonar.  An  undergraduate  Physics  student,  Raffaele 
Tavarone  from  Rome  University  worked  in  the  Pi’s  lab  during  summer  2009  to  analyze  flocking 
trajectory  data  for  starlings,  using  algorithms  developed  under  the  current  ARO  project.  Results 
so  far  are  very  promising,  yielding  insight  into  the  interaction  structure  of  the  flock. 


Collaborations  and  Leveraged  Funding 


A  key  collaboration  was  initiated  with  colleagues  Timothy  Horiuchi  (Associate  Professor, 
Electrical  and  Computer  Engineering  and  Institute  for  Systems  Research),  and  Cynthia  Moss 
(Professor  of  Psychology,  affiliated  with  Biology,  and  member  of  the  Institute  for  Systems 
Research),  and  postdoctoral  fellow  Kaushik  Ghose.  Further,  the  collaboration  with  Eric  Justh 
continues  (he  is  now  on  the  staff  of  the  Tactical  Electronic  Warfare  Division  of  the  Naval 
Research  Laboratory). 

The  collaboration  with  Moss  and  Horiuchi  was  supported  by  leveraged  funding  from  NIH- 
NIBIB  under  the  Collaborative  Research  in  Computational  Neuroscience  program  (CRCNS: 
Innovative  technologies  inspired  by  biosonar).  Separately,  there  is  an  AFOSR  3  year  project 
“Pursuit,  Avoidance,  and  Cohesion  in  Flight:  Multipurpose  Control  Laws  and  Neuromorphic 
VLSI  Implementation,”  under  the  auspices  of  the  Directorate  of  Life  Sciences,  Information  and 
Mathematics,  that  started  on  June  1  2007.  This  project  enables  collaboration  on  fundamental 
questions  pertaining  to  biologically  plausible  implementation,  and  in-silico  implementation.  The 
investigators  on  this  project  are  Horiuchi  (PI)  and  Krishnaprasad  (Co-PI).  A  new  collaboration 
with  a  group  of  faculty  from  the  Physics  and  Mathematics  Departments  in  Maryland  has  been 
initiated  and  funded  under  the  ONR  MURI  Program  (Exploiting  Nonlinear  Dynamics  for  Novel 
Sensor  Networks),  with  funding  begun  May  1,  2007.  Under  this  project,  Krishnaprasad  has  also 
completed  work  on  stochastic  control  of  free  space  optical  communication  links  with  his  Ph.D. 
student  Arash  Komaee  and  colleague  Prakash  Narayan.  This  work  on  closed-loop  control  using 
point  process  detection  models  and  nonlinear  filters  is  expected  to  be  useful  in  applications 
involving  line-of-sight  optical  links  for  coordination  of  groups  of  ground  and  aerial  vehicles. 

This  line  of  work  was  initiated  during  a  final  year  of  funding  under  an  ARO-MURI  program 
begun  in  May  2001,  on  Communicating  Networked  Control  Systems  (lead  institution:  Boston 
University).  Additional  leveraged  funding  provided  through  a  University  of  Maryland  initiative 
supports  some  of  the  work  with  Cricket  localization  system. 

Support  for  undergraduate  researchers  was  also  obtained  from  NSF  through  summer  research 
programs  during  summer  2006-2009.  A  group  of  summer  undergraduate  researchers  worked  on 
demonstrating  the  use  of  Cricket  location  sensing  system  to  guide  a  pair  of  robots  to  rendezvous 
using  our  2D  pursuit  law  in  a  cooperative  manner.  We  continue  to  involve  two  undergraduates  in 
the  Intelligent  Servosystems  Laboratory  on  problems  of  pursuit  and  collision  avoidance.  Ennin 
Wei  obtained  new  data  on  collision  avoidance  behavior  in  echolocating  bats  and  Philip  Twu 
studied  cyclic  pursuit  in  which  pursuit  laws  are  used  to  achieve  cohesion  (rendezvous).  Both 
have  graduated  in  May  2008  and  moved  on  to  conduct  doctoral  studies  (Wei  at  MIT,  and  Twu  at 
Georgia  Tech). 

A  new  collaboration  with  a  group  of  physicists  in  Rome,  Italy  studying  collective  behavior  in 
starlings  has  been  initiated,  including  joint  analyses  of  empirical  data.  This  group  is  engaged  in 
fundamental  experimental  and  analytical  studies  of  collective  behavior  under  funding  from  the 
European  Union. 


Conclusions 


Significant  progress  has  been  made  in  the  project,  on  both  theoretical  aspects  and  collaborative 
experimental  efforts.  The  success  in  investigating  delay  and  stochasticity  has  been  encouraging. 
These  are  key  issues  of  broad  interest  to  the  field.  The  funding  from  ARO  on  this  project  has 
been  crucial  in  providing  the  PI  time  to  develop  the  results. 

Technology  Transfer 

Graduate  student  Kevin  Galloway  (reserve  officer  in  the  US  Navy)  has  played  an  active  part  in 
Technology  Transfer,  by  serving  as  a  summer  2007  intern  at  the  Naval  Research  Laboratory 
(NRL),  working  on  issues  of  implementation  of  pursuit  and  cohesion  laws  in  unmanned  aerial 
vehicles,  working  with  a  distributed  hardware-in-the-loop  simulation  test-bed.  (From  January 
2009,  to  February  2010  he  was  on  leave,  following  call-up  for  active  duty,  in  Iraq.).  Additional 
technology  transfer  is  being  made  via  internships  for  students  at  Intelligent  Automation  Inc. 

(IAI),  a  company  located  in  Rockville,  MD,  with  significant  involvement  in  DOD  research  on 
swarming  and  distributed  systems.  The  internships  have  been  set  up  as  a  formal  collaboration 
and  technology  transfer  mechanism  between  the  Pi’s  lab  and  IAI.  Graduate  student  Matteo 
Mischiati  (advisee  of  the  PI)  and  undergraduate  student  Philip  Twu  both  spent  summer  2007  at 
IAI.  Mischiati  worked  on  path  planning  algorithms  for  a  team  of  mobile  robots  to  demonstrate 
mixed-initiative  intruder  stand-off  strategies.  Philip  Twu  was  engaged  in  bringing  to  completion 
a  new  distributed  agent-based  implementation  of  MDLe  for  open  source  release.  MDLe  was 
invented  in  the  Pi’s  lab  and  was  given  a  first  implementation  there.  The  new  industrial- strength 
implementation  at  IAI  is  built  on  a  distributed  agent  framework  CybelePro.  Graham  Alldredge 
was  intern  at  IAI  during  summer  2008,  working  on  robotics  projects  using  the  new 
implementation  of  MDLe.  He  taught  the  undergraduates  in  the  Pi’s  lab  during  summer  2009  the 
basics  of  this  technology  as  needed  for  their  experiments. 

Future  Plans 

The  PI  plans  to  extend  the  dyadic  pursuit  laws  that  have  been  developed  so  far  into  building 
blocks  for  cohesion  in  collectives  of  systems  (UAVs,  UGVs).  Further  work  on  delay  and 
stochasticity  as  well  as  new  efforts  on  the  side  of  optimal  control  and  game  theory  will  be  carried 
out.  The  collaboration  with  Professor  Moss  will  be  extended  to  include  multiple  bat  behavior  in 
the  laboratory.  The  PI  plans  to  investigate  additional  opportunities  for  technology  transfer 
through  Anny  labs.  The  PI  will  also  continue  collaborations  with  the  Italian  group  on  flocking. 
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